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DESCRIPTION 

SYSTEM FOR ENCODING OR DECODING MOTION PICTURE AND 
METHOD FOR ENCODING OR DECODING MOTION PICTURE 



Technical Field 

This invention relates to a motion picture encoding or decoding system and a 
motion picture encoding or decoding method by using a processor capable of changing an 
operating frequency and an operating voltage. 



Background Art 

In recent years, it has become possible to transmit and receive motion pictures via 
a transmission line and store the motion pictures in a storage medium. Since the motion 
pictures are generally large volumes of information, the technique for encod.ing/decoding 

15 such motion pictures constitutes an indispensable requirement when the motion pictures 
are transmitted by using a transmission line having a limited transmission bit rate or when 
tiie motion pictures are stored in a storage medium having a limited storage capacity. As 
methods for encoding/decoding motion pictures, MPEG (Moving Picture Experts Group) 
and H.26X whose standardization has been advocated by ISO/IEC, are available at present. 

20 They are directed toward encoding or decoding a plurality of chronologically continuous 
frames constituting such motion pictures and are intended as techniques for decreasing 
volumes of information of motion pictures by effecting a cutback of the redundancy 
utilizing the temporal correlation and the spatial correlation of motion pictures, encoding 
the resultant motion pictures, and again decoding the encoded motion pictures to the 

25 former motion pictures. 

These encoding/decoding techniques have been applied to such information 
terminals as personal computers and portable telephones using built-in microcomputers. 
Based on programs describing the procedures of encoding/decoding, these 
microprocessors function as a motion picture encoding system when the motion pictures 

30 are transmitted or as a motion picture decoding system when the motion pictures are 
received. Since the motion picture encoding or decoding process entails a comparatively 
large amoxmt of computation power and consequently tends to consume large amount pf 
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electric power, the effort directed toward, lowering the electric power consumption in the 
encoding and decoding process by using software haying a greater flexibility than 
hardware has become a major task. . 

Now, the conventional means for lowering the electric power consumption in the 
5 motion picture encoding or decoding system resorting to use of software will be explained 
below. The conventional means for lowering the electric power consumption are 
disclosed, for example, in the following Non-Patent Document 1 . 

(Non-Patent Document 1) Glossary of Abstracts, pp 918-921 "An LSI for VDD- 
Hopping and MPEG4 System Based on the Chip" (H. Kawaguchi, G. Zhang, S. Lee, and 

10 T. Sakurai) of IEEE International Symposium on Circuits and System 2001 (May, 2001). 

Fig. 10 is a diagreun showing a conventional method for effecting reduction of 
electric power consumption with respect to the motion picture encoding system disclosed 
in Non-Patent Document 1. Incidentally, the means of lowering the electric power 
consumption applies similarly to the motion picture decoding system. 

1 5 This example of the conventional method concerns a method for controlling the 

operating voltage and the operating frequency for implementing the decrease of the 
electric power consumption in encoding motion pictures (particularly MPEG) on the 
processor which is capable of dynamically changing the operating voltage and the 
operating frequency. To be specific, this example of the conventional method notices the 

20 fact that, during the operation of encoding motion pictures, the computation workload 
required for encoding or decoding motion pictures is varied every frame as by the activity 
and the intensity of motion within motion pictures as illustrated in Fig. 1 1 and attempts to 
attain the decrease of the electric power consumption by controlling the operating 
frequency and the operating voltage of the processor. 

25 In the operation of encoding, the time for processing one frame is restricted as by 

the specification of the encoding metiiod (such as MPEG) to the duration of T f and the 
operation of encoding one frame is required to be completed within this duration T f This 
duration T f (seconds) for processing one frame is divided into even N pieces. The 
individual pieces (time) are defined as a time slot T slot (T slot = T fiTsT) and the residual 

30 duration TRi is defined as TRi = Tf - Tslot *i. The nxmiber of blocks of motion, pictures to 
be processed in one time slot T slot (the motion pictures encoded by block units) is 
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denoted by R (R*N, therefore, indicates the nximber of blocks in one frame) and the time 
which is spent in processing (R*i) blocks (namely the time spent in the actual processing 
of the group of blocks to be processed from the time slot T slot 1 through the time slot T 
slot i f) is denoted by T acc (i+1). The time required for stabilizing the operating voltage 
5 and the operating frequency after chaing the operating voltage is denoted by T rd. The 
real time slot Rt slot i denotes the processing time actually required for the processing to 
be completed within the time slot T slot i. Li Fig. 10, the processing of the group of 
blocks allocated to the tune slot T slot 1 and the time slot T slot 2 is actuated by the clock 
frequency f max which enables the processing to be satisfactorily completed within the 

10 time slot T slot 1 and the time slot T slot 2 even when the workload for processing the 
blocks is maximal. When the time T acc 3 consumed for this processing satisfies this 
expression, T acc 3 < (T f - Tr 2), namely when the processing of the group of allocated 
blocks is completed within the time slots T slot 1 and T slot 2, the processing time T tar 3 
which is usable for processing the group of blocks allocated to the next time slot T slot 3 

1 5 satisfies the expression, Ttar3=Tf-Tacc3-TR3 - Trd. Since the processing of the 
group of blocks allocated to T slot 3 is only required to be completed within the 
processing time T tar 3, the operating frequency for the processing of this group of blocks 
is lowered. The processing times T fl, T f2, and T G shown in Fig. 10 denotes the 
durations of operation performed at the respective operating frequencies f 1, f 2, and f 3 

20 when the workload is maximal at the time slot T slot 3. When the operating frequency f 2 
= f max/2 shown in Fig. 10 is selected as the operating frequency, the processing even 
when the workload is maximal does not intrude into the next time slot T slot 4 in which 
the processing time ought to expire within the period extending from the time slot T slot 1 
through the time slot T slot 3 is within (T f - TR 3). When the operating frequency f 3 = f 

25 max/3 is selected instead, the processing time T f 3 inevitably exceeds the processing time 
T tar 3. For the group of blocks to be processed in this time slot T slot 3, therefore, the 
operation is performed at the operating frequency of f 2 = f max/2 and the operating 
voltage befitting the operating frequency. In the same manner, this process is repeated for 
each of the subsequent time slots T slots. 

30 When the operating clock frequency and the operating voltage are dynamically 

changed, therefore, the decrease of the electric power consumption is attained by selecting 
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the smallest operating frequency in all the operating frequencies which are capable of 
processing the group of prescribed number of blocks within the prescribed duration, 
enabling the operation to be performed with the operating frequency and the operating 
voltage synthetically lowered, and controlling the voltage as occasion demands. 
5 Incidentally, as regards the process (for example, the process in one frame) which 

ought to be completed within a stated processing time (such as, for example, the 
processing time T f for one frame here), the processor is preferably operated at fixed 
operating voltage and operating frequency throughout the entire processing time of the one 
frame. Specifically, on the assumption that the processing time for one frame is denoted 

10 by T f (second), the amount of operation by K f (cycle), and the operating frequency by F f, 
by setting the operating frequency at F f = K £^T f (cycles/second) and causing the 
processor to be operated at a fixed operating frequency F f throughout the entire length of 
the processing time Tf for one frame, it is made possible to attain the decrease of the 
electric power consumption as compared with the case of changing the operating 

1 5 frequency F f a number of times within the processing time T f. 

In the present example of the conventional method, however, notwithstanding the 
unit of synchronizing the processing time T f is one frame, the decrease of the electric 
power consumption has not been fully attained because the operating voltage and the 
operating frequency have been changed to the maximum of N times within one frame. 

20 That is, the decrease of the electric power consumption in the process for encoding and 
decoding motion pictures with a processor which is capable of controlling the operating 
voltage and the operating frequency in many stages as in the present example of the 
conventional method has required the operating voltage and the operating frequency to be 
changed a number of times during the processing of one frame. Since the processing time 

25 is restrained by the unit of a frame as described above, the process proceeding in one 
frame is preferred to be controlled with a prescribed lowest possible frequency which 
enables the process to proceed 

The present example of the conventional method in which the operating voltage and the 

> 

operating frequency are changed in the maximum of N times during the process of one 
30 frame, therefore, the decrease of the electric power consxmiption has not been fully 
attained. 
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This invention is directed toward solving the task mentioned above and 
consequently providing a motion picture encoding or decoding system and a motion 
picture encoding or decoding program which enables the electric power consumption to be 
copiously decreased as compared with the conventional technique mentioned above. 

5 

Disclosure of the Invention 

Specifically, the motion picture encoding or decoding system of this invention is 
characterized by being furnished with a processor for encoding or decoding motion 
pictures formed of a plurality of frames in the unit of frames and provided with at least 

10 one failure avoiding means to calculate a necessary volume Kp of operation required for 
encoding or decoding one frame, deciding an operating frequency F capable of encoding 
or decoding the necessary volume Kp of operation within a duration T e allocated in 
advance to the operation of encoding or decoding the one frame, performing the operation 
of encoding or decoding the one frame on a processor which operates at the operating 

15 frequency F and an operating voltage V befitting the operating frequency F, and avoiding 
the failure situation which occurs when the necessary volume Kp of operation is smaller 
than the actually necessary volume of operation. Then, the method of this invention for 
encoding or decoding motion pictures is characterized by being endowed with at least one 
failure avoiding step for calculating a necessary volume Kp of operation required for 
. 20 encoding or decoding one frame, deciding an operating frequency F capable of encoding 
or decoding the necessary volume Kp of operation within a duration T e allocated in 
advance to the operation of encoding or decoding the one frame, performing the operation 
of encoding or decoding the one frame on a processor which operates at the operating 
. frequency F and an operating voltage V befitting the operating frequency F, and avoiding 

25 the failure situation which occurs when the necessary volume Kp of operation is smaller 
than the actually necessary volume of operation by the use of a processor for encoding or 
decoding motion pictures formed of a plurality of frames in the unit of frames. 

In the specification of the encoding/decoding method (MPEG, for example), a 
processing time is allocated preparatorily to every frame. This invention calculates a 

30 necessary volume Kp of operation required for encoding or decoding one framCy decides 
an operating frequency F capable of encoding or decoding the necessary volume Kp of 
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operation within a duration T e allocated in advance to the operation of encoding or 
decoding the one frame, and performs the operation of encoding or decoding the one 
frame on a processor which operates at the operating frequency F and an operating voltage 
V befitting the operating frequency F. As a result, it is made possible to perform the 
5 encoding or decoding operation while enabling the processor to be constantly operated at 
the lowest possible operating voltage and operating frequency necessary for encoding or 
decoding each frame and consequently promoting the decrease of the electric power 
consumption as compared with the conventional technique which requires the operating 
voltage and the operating frequency to be changed a nimiber of times during the operation 

10 of encoding/decoding one frame because the operating frequency and the operating 
voltage are decided for each of the prescribed number of blocks formed by dividing the 
frame. Further, when the necessary volume Kp of operation is smaller than the actually 
necessary volume of operation, the failure situation which imparts a very bad quality to 
the images ensues because the encoding or decoding operation performed on the existing 

15 frame is not completed within the predetermined duration. This invention avoids the 
occurrence of the failure situation because it is fumished with at least one failure avoiding 
means capable of avoiding the failure situation. 

The motion picture encoding or decoding system/method according to this 
invention, in implementing the operation of encoding motion pictures on the assumption 

20 that the frame in the plurality of sequential frames which is subjected to the encoding 
operation prior to the aforementioned one frame is designated as the preceding frame, is 
characterized by calculating the necessary volimie Kp of operation by using at least one of 
the elements which comprise the amounts of motion of the aforementioned one frame and 
the preceding frame, the amoimt of activity of the aforementioned one frame, the amount 

25 of activity of the preceding frame, the average value of the quantized step size of the 
preceding frame, the difference between the average value of the quantized step size of the 
preceding frame and the average value of the quantized step size of the frame fiirther 
preceding the preceding frame, the number of times of macro block matching of the 
preceding frame, the number of valid blocks of the preceding frame, the volume of 

30 operation actually required in encoding the preceding frame, the nxmiber of bits occurring 
in the preceding frame, the encoding bit rate of the aforementioned one frame, the frame 
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encoding type selected between the intra frame encoding and the inter frame encoding to 
be performed on the aforementioned one frame, and the necessary volume of operation for 
the preceding frame. 

The motion picture encoding or decoding system/method according to this 
5 invention, in implementing the operation of decoding motion pictures on the assumption 
that the frame in the plurality of sequential frames which is subjected to the decoding 
operation prior to the aforementioned one frame is designated as the preceding frame, is 
characterized by calculating the necessary volume Kp of operation by using at least one of 

r 

the elements which comprise the number of bits of the encoding data of the 

10 aforementioned one frame, the frame encoding type selected between the intra frame 
encoding and the inter frame encoding which has been performed in the aforementioned 
one frame, the average value of the magnitude of motion vector of the aforementioned one 
frame or the preceding frame, the dispersion of the magnitude of the motion vector of the 
aforementioned one frame or the preceding frame, the number of valid blocks of the 

15 aforementioned one frame or the preceding frame, the number of valid coefficients of the 
aforementioned one frame or the preceding frame, the bit rate of the aforementioned one 
frame or the preceding frame, the amount of bit stream of the aforementioned one frame or 
the preceding frame, the average value of the quantized step size of the aforementioned 
one frame or the preceding frame, the difference of the average values of the quantized 

20 step size (the difference between the quantized step sizes of the aforementioned one frame 
and the frame further preceding the preceding frame or the difference between the 
quantized step size of the frame preceding by one frame and the quantized step size of the 
frame preceding by two frames), the necessary volume of operation actually required for 
decoding the preceding frame, and the necessary volume of operation for the preceding 

25 frame. 

The plurality of elements emmierated above is factors which affect the volume of 
operation to be performed in the encoding or decoding operation. According to this 
invention, since the necessary volume Kp of operation is calculated by using at least one 
of the elements mentioned above, the necessary volume Kp of operation foxmd by the 
30 calculation approximates closely to the volume of operation actually performed in the 
encoding or decoding operation. As a result, the incidence of the failure situation that the 
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necessary volume Kp of operation is smaller than the actual voliune of operation and the 
encoding or decoding operation is not completed within the specified duration can be 
decreased. Further, the possibility that the decrease of the electric power consumption 
will be inhibited because the calculated necessary volume Kp of operation is far larger 
than the actual volimie of operation is diminished. 

The motion picture encoding or decoding system/method according to this 
invention is characterized by being furnished, as the failure avoiding means/step 
mentioned above, with a first failiire avoiding means/step capable of increasing the 
necessary volume Kp of operation. 

According to this invention, since the failure avoiding means is capable of 
increasing the necessary volxune of operation by a prescribed value, the possibility that the 
calculated necessary volume Kp of operation will meet the actual volume of operation is 
heightened and the failure situation can be avoided. 

The first failure avoiding means/step mentioned above is characterized by 
multiplying the necessary volume Kp of operation by m (m denotes a real number of not 
less than 1) or adding a real number n larger than 0 to the necessary volume Kp of 
operation. 

According to this invention, since the first failure avoiding means/step is capable 
of multiplying the necessary volume Kp of operation by m or adding n to the necessary 
volume Kp of operation, it is made possible by adjusting the value of m or n to allow the 
calculated necessary amount Kp of operation to exceed the actual volume of operation or 
approximate closely to the real volume of operation and consequently avoid the failure 
situation. 

The motion picture encoding or decoding system/method according to this 
invention is characterized by being fiimished, as the failure avoiding means/step 
mentioned above, with a second failure avoiding means/step which, when the operation of 
encoding the aforementioned one frame is not completed within the duration allocated in 
advance to the operation of encoding or decoding the aforementioned one firame, performs 
the operation of avoiding the failure situation. 

In the first failure avoiding means/step mentioned above, the operation of avoiding 
the failure situation by increasing the necessary volume Kp of operation is implemented 
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without reference to the presence or absence of the failure situation. According to this 
invention, since the second failure avoiding means/step performs the operation of avoiding 
the failure situation when the operation of encoding the aforementioned one frame is not 
completed within the duration allocated in advance to the operation of encoding or 
5 decoding the aforementioned one frame, the failiure situation can be efficiently avoided 
because the operation of avoiding the failure situation is carried out only when failure 
situation occurs. 

The second failure avoiding means/step mentioned above is characterized by 
interrapting the encoding operation at a prescribed timing and, on the occurrence of 

10 macro blocks which have not xmdergone the encoding operation, performing a forcedly 
not-coded block operation on the macro blocks. In the decoding operation, the forcedly 
not-coded block operation is not performed. 

When the macro blocks which have not undergone the encoding operation occur, 
for example, in the prescribed timing which remains after subtracting the processing time 

15 for the processing all the macro blocks as forcedly not-coded blocks from the duration T e 
allocated in advance to the operation of encoding one frame, the possibility that the failure 
situation will occur is high because the calculated necessary volume of operation is 
smaller than the actually necessary volume of operation. According to this invention, the 
failure situation can be avoided because die second failure avoiding means/step interrupts 

20 the operation of encoding or decoding motion pictures at the aforementioned timing, for 
example, and, in the occurrence of a macro block which has not undergone the encoding 
operation, judges that the operation of encoding the aforementioned one frame will not be 
completed within the duration T e and the failure situation will consequently ensue, and 
performs the processing the relevant macro blocks as forcedly not-coded blocks. 

25 The second failure avoiding means/step mentioned above is characterized by being 

provided with at least an operation residue judging means which interrupts the encoding or 
decoding operation at a prescribed timing and, when the residual portion of the necessary 
volume Kp of operation at the time of the intermption is smaller than the residual portion 
of the necessary volimie of operation actually required for the encoding or decoding 

30 operation, increases the operating frequency and the operating voltage of the processor. 

When the residual portion of the calculated necessary volume Kp of operation is 
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smaller than the residual portion of the necessary volume of operation actually required 
for the encoding or decoding operation halfway along the coxu"se of the encoding or 

9 

decoding operation, the possibility that the failure situation will occur is high because the 
calculated necessary volume Kp of operation mentioned above is smaller than the 
5 necessary volume of operation actually required for the encoding or decoding operation. 
Thus, the second failure avoiding means/step is rehed on to interrupt the motion picture 
encoding or decoding operation at the prescribed timing and to compare the residual 
portions of necessary volume of operation mentioned above. When the residual portion of 
the necessary volume Kp of operation is judged by the comparison to be smaller than the 

10 residual portion of the necessary volume of operation actually required, the means/step 
mentioned above judges that the encoding or decoding operation will not be completed 
within the duration T e and the failure situation will consequently ensue and increases the 
operating frequency and the operating voltage of the processor. Since the speed of the 
operation by the processor is consequently heightened, it is made possible to increase the 

15 volume of operation available and avoid the failure situation. By pluralizing the number 
of interruption, it is made possible to increase the operating frequency and the operating 
voltage stepwise in conformity with the state of processing and add fiuther to the 
possibility of avoiding the failure situation. 

The processor mentioned above is characterized by having the r stages (r denotes 

20 an integer number of not less than 2) operating frequency available for the processing 
operation, calculating the operating frequency F e necessary for dealing with the necessary 
volume Kp of operation within the aforementioned duration T e in accordance with the 
formula F e = Kp/Te, and deciding from the operating frequency available for the 
operation of the aforementioned processor the operating frequency F exceeding the 

25 necessary operating frequency F e mentioned above and approximating lost closely to the 

operating frequency F e. 

According to this invention, the operating frequency F e necessary for dealing with 
the necessary volume Kp of operation within the duration Te is calculated by the formula 
F e = Kp/Te, then the operating frequency F exceeding the necessary operating frequency 
30 F e mentioned above and approximating most closely to the operating frequency F e is 
decided from the operating frequency available for the operation of the processor, the 



11 



operating voltage V befitting the decided operating jfrequency F is decided as well, and the 
processor is operated at the operating frequency F and the operating voltage V to effect the 
encoding or decoding operation. That is, since the processor is operated at the smallest 
operating frequency F and operating voltage V available for dealing with the necessary 
5 volume Kp of operation within the duration T e in all the operating frequency and 
operating voltage available for the operation of the processor and the aforementioned 
operation of encoding or decoding the aforementioned one frame is processed on this 
processor, the decrease of the electric power consumption is efficiently accomplished even 
when the processor to be used happens to have the available operating frequency prepared 
10 stepwise. 

Brief Description of the Drawing 

Fig. 1 is a schematic block diagram illustrating the performance of a motion 
picture encoding system in the first mode of embodiment of this invention. Fig. 2 is a 

1 5 diagram illustrating an example of implementing the motion picture encoding system of 
the mode of embodiment mentioned above. Fig. 3 is a diagram illustrating a motion 
picture encoding program for functionalizing a computer as a motion picture encoding 
system in the mode of embodiment mentioned above and a schematic flow chart of a 
motion picture encoding method realized by the program. Fig. 4 is a diagram showing the 

20 relation between the time for encoding operation and the residual portion of the operation 
in the motion picture encoding system in the mode of embodiment mentioned above. Fig. 
5 is a conceptual diagram illustrating the operating voltage and operating frequency of a 
processor to be used in the motion picture encoding system in the mode of embodiment 
mentioned above. Fig. 6 is a diagram for aiding in the explanation of the availability of 

25 this invention. Fig. 7 is a schematic block diagram illustrating the performance of a 
motion picture encoding system in the second mode of embodiment of this invention. Fig. 
8 is a schematic flow chart of a motion picture encoding program for fimctionalizing a 
computer as a motion picture encoding system in the mode of embodiment mentioned 
above. Fig. 9 is a schematic block diagram illustrating the performance of a motion 

30 picture decoding system in the third mode of embodiment of this invention. Fig. 10 is a 
diagram illustrating a conventional method for effecting a decrease in the electric power 
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consumption with respect to the motion picture encoding system. Fig. 1 1 is a conceptual 
diagram illustrating the state in which the volxmie of operation for encoding or decoding 
motion pictures is varied by the unit of frames. 

§ 

5 Best Mode of Embodying the Invention 

Now, the motion picture encoding or decoding system and the method for 
operating the system according to this invention will be described as separated into the 
encoding operation and the decoding operation in detail below. 
[First mode of embodiment] 

10 The motion picture encoding system in the first mode of embodiment of this 

invention is realized by a computer which is an information terminal such as a portable 
telephone having a microprocessor built therein or a personal computer. Particularly, it is 
a system which is functioning in the computer as part of a multimedia signal processing 
part and it is a system for sequentially encoding motion pictures formed of a prescribed 

1 5 number of continuous frames in the imit of frames. 

Fig. 1 is a schematic block diagram illustrating the performance of a motion 
picture encoding system SI in the present mode of embodiment. The motion picture 
encoding system S 1 is a computer (particularly a multimedia signal processing part in a 
computer) provided at least with a processor 1 having an operating voltage and an 

20 operating frequency prepared in r stages (r denotes an integer of not less than 2) (namely, 
enabled to operate with r stages of operating voltage and operating frequency) and enabled 
to vary the operating voltage and the operating frequency by a program, an operating 
voltage and operating frequency controlling means 4 fiimished with a DC-DC converter, a 
PLL, etc. and adapted to control the operating voltage and the operating frequency of the 

25 processor 1, a local decoded ft^me memory 6 which is a memory region for storing 
prescribed data, an input frame memory 7, an element memory 8, and a processed macro 
block number register 10. Only, the local decoding memory 6 and the input frame 
memory 7 may have the operating voltage and operating frequency thereof controlled by 
the operating voltage and operating frequency controlling means 4 in the same manner as 

30 the processor 1. In the present mode of embodiment, the components contained in a 
controlling region CA indicated with a dotted line (the processor 1, the local decoded 
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frame memory 6, the element memory 8, the processed macro block number register 10, 
input frame memories 7a and 7b, etc.) have tiie operating frequency and the operating 
voltage thereof controlled. 

The processor 1 is provided, as means operating on the processor 1 , with a means 2 
5 to calculate the necessary volume of operation, an operating voltage and operating 
frequency calculating means 3, a motion picture encoding means 5, and two failure 
avoiding means 9 and 1 1 . The two failure avoiding means 9 and 1 1 are means to avoid 
the failure situation which occurs when the necessary volume of operation calculated by 
the means 2 for calculating the necessary volume of operation is smaller than the volume 

10 of operation actually necessary for the encoding operation performed by the encoding 
means 5. Specifically, they are the first failure avoiding means 11 functioning as part of 
the means 5 for calculating the necessary volume of operation and the means 9 for judging 
completion of the operation as the second failure avoiding means. Incidentally, the 
reference nximeral 101 denotes an input image data, the reference numeral 102 denotes an 

15 indicator of operating voltage and operating frequency, the reference numeral 103 denotes 
a local decoding data of the preceding frame, the reference numeral 105 denotes an 
operating voltage and operating frequency supply, the reference nimieral 106 denotes a 
frame encoding data, the reference numeral 107 denotes an information of the average 
value of the quantized step size of the preceding frame, the reference numeral 1 08 denotes 

20 the frame encoding type of each frame to be selected between the intra frame encoding 
and the inter frame encoding, the reference numeral 109 denotes the information of the 
encoding bit rate of motion pictures, the reference numeral 110 denotes the amoimt of 
activity of the preceding frame, the reference numeral 111 denotes the number of times of 
macro block matching of the preceding frame, the reference numeral 112 denotes the 

25 number of valid blocks of the preceding frame, the reference numeral 113 denotes the 
number of valid coefficients of the preceding fi^me, the reference numeral 114 denotes the 
difference between the average value of the quantized step size of the preceding frame and 
the average value of the quantized step size of the frame preceding the preceding frame by 
one frame, the reference muneral 115 denotes the volume of operation actually required 

30 for encoding the preceding frame, the reference numeral 116 denotes the necessary 
volume of operation of the preceding firame calculated by the means 2 for calculating the 



necessary voliune of operation, and the reference numeral 117 denotes the number of 
processed macro blocks which is the number of macro blocks having completed the 
encoding operation. The element memory 8 is a memory region for storing part of the 
plurality of elements to be used in the means 2 for calculating the necessary volume of 
5 operation as specifically described herein below (the frame encoding type 1 08 of each 
frame selected between the intra frame encoding and the inter frame encoding, the 
encoding bit rate 109, the amount 1 10 of activity of the frame, and the necessary volume 
116 of operation calculated by the means 2 for calculating the necessary volume of 
operation). The processed macro block number register 10 is a register for storing 

10 temporarily the information of the number 1 17 of macro blocks which have undergone the 
encoding operation. The motion picture encoding means 5 uses MPEG-4 for the method 
of encoding. It may optionally use other method of encoding such as, for example, H.26X, 
MPEG-1, orMPEG-2. 

Fig. 2 illustrates an example of implementing the motion picture encoding system 

15 SI. The system SI is realized by hardware which is furnished mainly with the processor 1, 
such peripheral units as various memories MR, 7a, and 7b, and various interfaces CI, Dl, 
and Bl, and an operating voltage and operating frequency controlling circuit 4a The 
various component elements enumerated above are rendered mutually communicable 
through flie medium of buses Bl and B2. 

20 The processor 1 is provided with a processor core la, an instruction cache memory 

lb, and a data cache memory Ic. The means 2 for calculating a necessary volume of 
operation, the means 3 for deciding the operating voltage and operating frequency, the 
means 5 for encoding motion pictures, and the means 9 and 1 1 for avoiding failure are 
realized by the optional execution on the processor core la of the programs stored in the 

25 memory MR. The instruction cache memory lb and the data cache memory Ic are such 
cache memories which are provided for the purpose of expediting the operation of the 
programs on the processor core la. 

The local decoded frame memory 6, the element memory 8, and the processed 
macro block number register 10 are collectively stored in the memory MR shown in Fig. 2. 

30 Meanwhile, the average value 107 of the quantized step size of the preceding frame, the 
frame encoding type 108 of each frame selected between the intra frame encoding and the 
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inter frame encoding, the motion picture encoding bit rate 109, the amoimt 1 10 of activity 
of the preceding frame (the past frame), the number 1 1 1 of times of macro block matching 
of the preceding frame, the number 112 of valid blocks of the preceding frame, the 
number 113 of valid coefficients of the preceding frame, the value 114 of the difference 
5 between the average value of the quantized step size of the preceding frame and the 
average value of the quantized step size of the frame preceding the preceding frame by 
one frame, the amoimt 115 of operation actually required for encoding the preceding 
frame, the necessary volume 1 1 6 of operation of the preceding frzme calculated by the 
means for calculating the necessary volume of operation, and the processed macro block 

10 number 1 17 are stored as data in a memory If The local decoded data 103 are transmitted 
and received as signals lOOj, 100k, and 1001 through the medium of a bus controller BC 
between the memory MR and the processor core la. 

The two input firame memories 7a and 7b correspond to the frame memory 7 
shown in Fig. 1. The video data (input image data 101) input from a camera interface CI 

15 are input through the medium of a bus B2 into the input frame memory 7a (or the input 
frame memory 7b). The input frame memory (#0) 7a and the input frame memory (#1) 7b 
exchange their services whenever one frame has been processed- That is, when the process 
in the i-th frame causes the input image data to be written by the signal lOOh in the input 
fi^me memory (#1) 7b and the input image data to be read out by the signal lOOo from the 

20 input frame memory (#0) 7a for the sake of the operation of encoding by the motion 
picture encoding means, the process in the (i + l)th frame causes the input image data to 
be written by the signal 100 i into the input frame memory (#0) 7a and the input image 
data to be read out by the signal lOOp from the input frame memory (#1) 7b for the sake of 
the operation of encoding by the motion picture encoding means. Thus, the signal lOOp 

25 does not occur when the input image data is being written by the signal lOOh into the input 
frame memory (#1) 7b and the signal lOOh does not occur conversely when the image is 
being read out by the signal lOOp. Likewise, the signal lOOo does not occur when the 
input image data is being written by the signal 1 OOi into the input frame memory (#0) 7a 
and the signal 1 OOi does not occur when the input image data is being read out by the 

30 signal lOOo from the input firame memory (#0) 7a. In this case, the input frame memory 
(#0) 7a in the process of the i-th frame and the input frame memory (#1) 7b in the process 
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of the (i + l)th frame respectively constitute the subject for control of the operating 
frequency and the operating voltage. By having the input frame memory prepared one 
each for two frames and enabling the respective operating frequencies to be set 
independently as described above, it is made possible to. execute the action of writing the 
input image data from the camera interface CI which constantly uses a fixed operating 
frequency and the action of reading the input image data having the operating frequency 
varied based on the calculated value of the necessary volume of operation without being 
mutually interrupted. 

The operating voltage and operating frequency controlling circuit 4a is enabled to 
effect mutual transmission and reception of a signal with PLL 4b and DC-DC converter 4c 
and these components collectively function as the operating voltage and operating 
frequency controlling means 4. The operating voltage and operating frequency controlling 
means 4 receives the operating voltage and operating frequency designation 102 in 
response to the signal lOOe from the processor core la and transmits the signal lOOu to the 
PLL 4b and the signal lOOv to the DC-DC converter 4c based on the designation 102. The 
PLL 4b transmits the operating frequency signal 100a based on the signal lOOu and the 
DC-DC converter 4c supplies the operating voltage lOOb based on the signal lOOv. As a 
result, the operating frequency and the operating voltage are controlled on the part of the 
components included in the control region CA indicated by a dotted line in Fig. 2 
(processor 1, memory MR, input frame memories 7a and 7b, bus controller BC/etc). The 
signals lOOe, lOOj, 100k, 1001, lOOm, lOOo, lOOp, lOOq, lOOr, and 100s have the 
frequencies and the signal levels thereof varied in conformity with the operating frequency 
signal 100a emitted by the PLL 4b and the value of the supply voltage 100b emitted by the 
DC-DC converter 4c. 

The encoded data which resxilts from the encoding by the motion picture encoding 
means 5 operating on the processor 1 is transmitted as the signal 100m via the bus Bl to a 
bit stream interface BI and output as the signal 1 OOn and transmitted to the memory MR 
ftmctioning as the local decoded frame memory 6. Then, the data of images is read out as 
the signal lOOq from the memory via the bus Bl and transmitted to a display interface DL 
The signal lOOq which has been received by the display interface DI is fed out as the 
video data by the signal lOOt. The video data is output and displayed as motion pictures 
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through the medium of a monitor connected to the display interface DI. 

The operating voltage and operating frequency controlling circuit 4a, the display 
interface DI, and the bit stream interface BI are constantly operated with fixed operating 
voltages. The signals lOOe, lOOq, and 100m which are transmitted and received between 
5 these components have the signal levels thereof varied in conformity with the changes of 
operating voltage of the components (processor 1 , memory MR, input frame memories 7a 
and 7b, etc.) contained in the controlling region CA. For the purpose of absorbing this 
influence, the operating voltage and operating frequency controlling circuit 4a, the display 
interface DI, and the bit stream interface BI are preferred to be provided with a level 

10 converter which corrects the signal levels of the received signals lOOe, lOOq, and 100m. 

Now, the performance of the motion picture encoding system SI in the present 
mode of embodiment will be explained below with reference to Fig. 1. The motion 
picture encoding system S 1 is realized by causing a computer (particularly the multimedia 
signal processing part in a computer) to fiinction as the following prescribed means by the 

1 5 motion picture encoding program Prg 1 . On the assumption that one arbitrary frame now 
on the verge of being encoded in all the frames to be sequentially encoded (that is, the 
frame which will be encoded next based on the point at which a certain frame is encoded, 
i.e. the frame which has not yet been encoded by that point and is expected to be encoded 
in the ftiture) is designated as the present frame and one frame which has been encoded 

20 before the present frame (the frame which was encoded in the past) as the former frame, 
the process of encoding the present frame will be explained below. The same process is 
carried out on any of the frames. 

Fig. 3 is a diagram showing a schematic flow chart of the motion picture encoding 
program Prg 1 . At step 1 to step 5 which will be specifically described herein below, the 

25 computer is fimctioned as the following individual means: (Step 1) for inputting the 
image data in the present frame into the input frame memory 7. (Step 2) for causing the 
computer to fimction as the necessary operation volume calculating means 2 for 
calculating the necessary volume Kp of operation for the present frame. (Step 3) for 
causing the computer to fimction as the operating voltage and operating frequency 

30 calculating means 3 for calculating the operating frequency F and the operating voltage V 
of the processor according to the calculated necessary volume Kp of operation. (Step 4) 
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for causing the computer to function as the operation voltage and operating frequency 
controlling means 4 for controlling the operation of the processor 1 at the calculated 
operating frequency F and operating voltage V. (Step 5) for causing the computer to 
function as the motion picture encoding means 5 for encoding the image information of 
5 the present frame. The motion pictures are encoded by causing the process from the step 1 
through the step 5 mentioned above to be performed on all the relevant frames 
sequentially in the order the frames are input in the input frame memory 7 (namely in the 
order in which the frames are encoded). The process of encoding will be explained in 
detail below. . 

10 (Step 1) The input image data which has been fed in is temporarily stored, for the 

purpose of synchronizing frames, in the input frame memory 7 which is the memory 
region for temporarily storing frames. 

(Step 2) The means 2 for calculating the necessary volume of operation gains 
access to the input frame memory 7, acquires the input image data 1 0 1 of the present 

1 5 frame, and calculates the necessary volume Kp of operation required for the operation of 
encoding the present frame. While various methods are conceivable for the calculation of 
the necessary volume Kp of operation, it is preferable to implement the calculation, for 
example, by using not less than one element which affects the volume of operation to be 
used in the operation of encoding the present frame. When the operation of encoding 

20 motion pictures is directed toward the compensation of motion and is consequently 
required to pay due attention to the fact that images of violent motion need a large volume 
of operation and images of moderate motion need a small volume of operation, the 
concrete examples of the element mentioned above include the value of strain calculated 
as the amount of movement of the present frame and the former frame by the sum of 

25 absolute difference value, the value calculated as the amoimt of activity of the individual 
frames by the sum of absolute difference values of adjacent picture elements, the number 
of times of macro block matching, the number of valid blocks, the number of valid 
coefficients, the encoding bit rate, the number of generated bits, the volume of operation 
actually required for encoding the former frame, and the necessary volume of operation of 

30 the former frame calculated by the means 2 for calculating the necessary volimie of 
operation. Here, on the assimiption that these elements are such that the value of only one 
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element varies and the values of the other elements do not vary, the necessary volimie of 
operation relatively increases when the value of that one element is large than when it is 
small and the necessary volxmie of operation decreases when the value of that one element 
is small than when it is large. Further, the necessary volume Kp of operation relatively 
5 decreases when the present frame undergoes intra frame encoding than when it undergoes 
inter frame encoding and the necessary volume Kp of operation relatively increases when 
the present undergoes inter frame encoding than when it imdergoes intra frame encoding. 
That is, since the plurality of elements affect the necessary volume of operation required 
for the encoding the present frame, the necessary volxune Kp of operation calculated by 

10 the means 2 for calculating the necessary volume of operation approaches closely to the 
volume of operation obtained by actually performing the operation of encoding when the 
means 2 for calculating the necessary volume of operation is caused to perform the 
calculation so as to increase or decrease the necessary volxmie Kp (cycle) of operation in 
compliance with these elements. 

15 For example, the present mode of embodiment performs the calculation by using 

the function G and forecasts (calculates) the size of movement of the input image by 
comparing the input image data 101 of the present frame stored in the input frame memory 
7 with the local decoded data 103 of the decoded former frame accumulated in the local 
decoded frame memory 6. The local decoded data 103 of the former frame is formed by 

20 the local decoder during the operation of encoding the former fiwe which is encoded 
prior to the present frame and is stored in the local decoded frame memory 6. Now, the 
method for determining the sxmi of absolute difference value 2 and the necessary volume 
Kp of operation will be explained below. Incidentally, as the image data of the former 
frame, the local decoding data 103 decoded by the local decoder after the encoding may be 

25 used or the input image data of the former frame which has been input may be used as it is. 

On the assumption that the input image data 101 of the present frame accmnulated 
in the input frame memory 7 is denoted by X (i, j) (wherein i denotes the coordinates of 
the image in the horizontal direction and j denotes the coordinates of the image in the 
vertical direction) and the local decoded data 103 of the former frame accumulated in the 

30 local decoded frame memory 6 which will be specifically described herein after is denoted 
by Y (i, j) (wherein i denotes the coordinates of the image in the horizontal direction and j 
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denotes the coordinates of the image in the vertical direction), the amoiints of motion of 
the present frame and the former frame will be determined by performing the calculation 
of the sum of absolute difference value Z =2:| X (i, j) - Y (i, j) | on all (or sampled) picture 
images. The value of this sum of absolute difference is denoted by Z. Meanwhile, the 
5 amount of activity of the frame is determined by calculating the sum of absolute difference 
value of adjacent picture element W in X (i, j), namely Wh in the horizontal direction =2| 
X (i, j) - X (i-1, j)| and Wv in the vertical direction =E| X (i, j) - X (i , j-1). This 
calculation is performed on all (or sampled) input images. The sum of absolute difference 
value of adjacent picture element (namely the amounts of activity of the individual 

10 frames) is denoted by W. 

Let Z stand for the sum of absolute difference value, Wa for the amount of activity 
of the present frame, Wb for the amount of activity of the former frame (the past frame), 
Qprev for tiie average quantized step size of the former frame (the average value of 
quantized step sizes), M for the number of times of macro block matching of the former 

15 frame, B for the number of valid blocks of the former frame, C for the number of valid 
coefficients of the former frame, S for the amoimt of processing actually required for 
encoding the former frame, BR for the encoding bit rate of the present frame, AQprev for 
the difference between the average value of the quantized step size of the former frame 
and the average value of the quantized step size of the frame preceding the former frame 

20 by one frame, D for the nxmiber of bits actually occurring in the encoding former frame, 
and Kp' for the calculated necessary volume of operation of the former frame, then the 
necessary volimie Kp of operation will be calculated by using not less than one of all these 
elements as follows. 

Kp = G (Z, Wa, Wb, Qprev, M, B, C, S, BR, AQprev, D, Kp') ... (Mathematical 1) 
25 wherein G denotes the function derived from at least one of the elements Z, Wa, Wb, 
Qprev, M, B, C, S, BR, AQpev, D, and Kp'. As one example, the following formula 

Kp = j + aM + PB + yC + 5Z + e AQprev ... (Mathematical 2) 
may be cited, though not exclusively. Then, as the element to be used for the calculation 
of the necessary volume Kp of operation, the frame encoding mode I of the present frame 
30 selected between the intra frame encoding and the inter frame encoding is used. The 
necessary volume Kp of operation assumes a small value when the present frame 
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undergoes the intra frame encoding and a large value when the frame xmdergoes the inter 
frame encoding. That is, the means 2 for calculating the necessary volume of operation, 
when using the sum of absolute difference value Z, calculates the sum of absolute 
difference differential value Z =S| Xij - Yij | and subsequently calculates the necessary 
5 volume of operation Kp = G (Z, Wa, Wb, Qprev, M, B, C, S, BR, AQpev, D, Kp')- 

Now, the function G mentioned above will be explained below. When the change 
of images between the former frame and the present frame is large (small), namely when 
the sum of absolute difference value Z is large (or small), the number of times of macro 
block matching executed in the present frame increases (decreases) and the volume of 

10 operation necessary for the operation of detecting the movement of the present frame 
(which depends on the number of times of macro block matching to be executed) is 
increased (decreased). When the amount Wa of activity of the present frame is large 
(small), this fact means that the present frame contains the high frequency component of 
images in a large (small) amount. In this case, the number of valid blocks and the number 

15 of valid coefficients which occur in the operation of encoding the present frame are 
^ increased (decreased) and the volume of operation required for the operation of subjecting 
the present frame to IDCT (which depends on the number of valid blocks to be generated), 
the volume of operation necessary for the IQ process (which depends on the nimiber of 
valid coefficients to be formed), and the volume of operation necessary for the VLC 

20 process (which depends on the number of valid coefficients to be formed) are increased 
(decreased). Thus, the function G mentioned above is so configured that the Kp may be 
set at a large (small) magnitude when such parameters as Z and Wa are large (small). 

Since the motion pictures have large correlations between the continuing frames, 
the number of times of macro block matching executed in the encoding operation, the 

25 number of valid blocks generated in the encoding operation, the number of valid 
coefficients generated in the encoding operation, the volume of operation required for the 
encoding operation, and the amoxmt of activity severally assume very close magnitudes 
between the frames continuing along the course of time. When M, B, C, S, and Wb are 
large (small), therefore, the probability that the number of times of macro block matching, 

30 the number of valid blocks, the number of valid coefficient, the volume of operation 
required for the encoding operation, and the amount of activity will increase (decrease) in 
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the present frame as well is high. Further, the relation S =s Kp' arises when the forecast 
volume of operation calculated by the means for calculating the necessary voliune of 
operation approximates closely to the volume of operation actually required for the 
encoding operation. Thus, the function G mentioned above is so configured that the Kp 
5 may be set at a large (small) magnitude when the parameters M, B, C, S, Wb, and Kp' are 
large (small). 

When the target bit rate is large (small), the quantized step size is set at a large 
(small) magnitude. As a result, the mmiber of valid blocks generated in the encoding 
operation and the number of valid coefficients are increased (decreased). When the 

10 number of bits generated in the former frame is large (small) as compared with the target 
bit rate, the value of the quantized step size of the present frame is set at a small (large) 
magnitude and the niunber of valid blocks and the number of valid coefficients which 
occur in the encoding operation decrease (increase). As a result, the function G mentioned 
above is so configured that the Kp may be set at a large (small) magnitude when tiie 

15 encoding bit rate BR of the present frame is large (small) and the Kp may be set at a small 
(large) magnitude when the bit rate BR is large (small) as compared with the number D of 
the bits actually generated in the former frame. Further, by taking into consideration the 
average quantized step size Qprev of the former frame and the difference AQprev between 
the average value of the quantized step size of the former frame and the average value of 

20 the quantized step size of the frame preceding the former frame by one frame, the Kp 
calculated by the function G mentioned above is enabled to approximate closely to the 
volume of operation actually required for encoding the present frame. 

(First failure avoiding step) For the purpose of rendering the failure situation 
difficult to occmr, the first failure avoiding means 11 included in the means 2 for 

25 calculating the necessary volume of operation is so processed that the necessary volume 
Kp of operation may be increased by a prescribed magnitude and the calculated necessary 
volume Kp of operation may enjoy an allowance. To be specific, the necessary volume 
Kp of operation is increased to m times the normal magnitude (m denotes an actual 
number of not less than 1). When m = 1.1, for example, is satisfied, the calculated 

30 necessary volxmie Kp of operation is enabled to enjoy an allowance of 10%. Alternatively, 
the necessary volxmie Kp of operation may be increased by the addition of a real nimiber n 



1 



23 



(n denotes a real number of not less than 0). This addition enables the necessary volume 
of operation to enjoy an allowance of a stated magnitude without reference to the the 
necessary volume of operation. By following the example cited above, the necessary 
volume Kp of operation which is finally calculated is determined in accord^ance with the 
5 following formulas. 

Kp = G (Z) * m ... (Mathematical 3) 

Kp = G (Z) + m ... (Mathematical 4) 

Alternatively, the following formula derived by combining these two formulas 
may be used. 

10 Kp = G (Z) * m + n ... (Mathematical 5) 

When the necessary volume Kp of operation foimd by the calculation is still 
smaller than the real necessary volume Km of operation of the present frame, the failure 
situation is avoided by resorting to the operation of a process completion judging means 9 
which is the second failure avoiding means as specifically described herein below. 

15 Incidentally, the encoding bit rate 109 of motion pictures, the frame encoding type 

108 for selecting the present frame and the former frame severally between the intra frame 
encoding and the inter frame encoding, the amount of activity 110 of the former frame, 
and the necessary volume 1 1 6 of operation of the former frame calculated by the means 
for calculating the necessary volxmie of operation have been memorized in advance in the 

20 element memory 8 which is a memory region for storing elements. They are read in the 
means 2 for calculating the necessary volume of operation during the calculation of the 
necessary volume Kp of operation and put to use. The average value 107 of the quantized 
step size of the former frame, the nxmiber 111 of times of macro block matching of the 
former frame, the number 112 of valid blocks of the former frame, the munber 113 of 

25 valid coefficients of the former frame, the difference 114 between the average value of 
quantized step size of the former frame and the average value of quantized step size of the 
frame preceding the former step by one frame, and the amount of process 115 actually 
required volxmie of operation for encoding the former frame are fed back from the means 
5 for encoding motion pictures to the means 2 for calculating the necessary volume of 

30 operation after the former frame has been encoded. In the means 2 for calculating the 
necessary volume of operation, only one of these elements may be used or a plurality of 
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such elements may be xised in combination. 

(Step 3) The means 3 for calculating the operating voltage and operating frequency 
-performs a calculation for forecasting the operating frequency Fe (cycle/second) for 
processing the present frame based on the value of the necessary volume Kp of operation. 
5 Since the minimiun unit for which the processing time is specified by the method of 
encoding is one fi^me and since the operating frequency Fe (cycle/second) required for 
the present frame, namely the operating frequency Fe (cycle/second) which enables the 
aforementioned necessary volume Kp of operation to be encoded within the duration Te 
(second), is expressed by the formula, Fe = Kp/Te, wherein Te (second) denotes the 

10 duration allocated to the operation of encoding the present frame, the means 3 for 
calculating the operating voltage and operating frequency performs the calculation of the 
operating frequency Fe = Kp/Te. It is provided, however, that the duration Te allocated to 
the operation of encoding the prescribed frame is the difference obtained by subtracting 
the duration Tp for forecasting the volume of operation for the prescribed frame and the 

15 duration Ts for varying the operating frequency and operating voltage of the processor 
from the restricted duration Tf for the operation of processing one frame. When the 
operating voltage and operating frequency supported by the peripheral unit including the 
processor 1 and (or) the local decoding memory 6 can be varied in r stages (r denotes an 
integer of not less than 2), the means 3 for calculating the operating voltage and operating 

20 frequency performs a calculation for selecting the operating frequency F (r) satisfying F 
(r) > Fe and F (r-1) < Fe as well as the operating frequency for performing the operation of 
encoding the present frame, performs a calculation for selecting the operating voltage V 
(r) befitting the operating frequency, and informs the means 4 for controlling the operating 
voltage and operating frequency in order that the peripheral unit including the processor I 

25 and (or) the local decoding memory 6 may be operated with the operating frequency F (r) 
and the operating voltage V (r) ( reference numeral 102). 

(Step 4) The means 4 for controlling the operating voltage and operating frequency 
supplies the values of the operating voltage V (r) and the operating frequency F (r) 
received from the means 3 for calculating the operating voltage and operating frequency 

30 to the peripheral xmit including the processor 1 and (or) the local decoding memory 6 
(reference numeral 105), and executes a control directed toward operating the processor 1 
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constantly at the operating voltage V (r) and the operating frequency F (r). As a result, the 
peripheral unit including the processor 1 and (or) the local decoding memory 6 is fated to 
operate constantly with the operating voltage V (r) and the operating frequency F (r). 

(Step 5) The means 5 for encoding motion pictures is a means which is realized on 
5 the processor 1 of the computer by the program Prg 1 for the operation of encoding motion 
pictures. This means accesses the input image data stored in the input frame memory 7 in 
the imit for encoding motion pictures by the use of the processor 1 and performs the 
operation of encoding the relevant input image data. That is, the means 5 for encoding 
motion pictures acquires the input image data 101 of the present frame from the input 

10 frame memory 7 and encodes the input image data 101 to give rise to the encoded data 
106. Since the peripheral xmit including the processor 1 and (or) the local decoding 
memory 6 was in the state of being operated constantly with the operating voltage V (r) 
and the operating frequency F (r) supplied from the means 4 for controlling the operating 
voltage and operating frequency in Step 4, the means 4 for controlling the operating 

15 voltage and operating frequency is destined to operate constantly the peripheral imit 
containing the processor 1 and (or) the peripheral decoding memory 6 with the operating 
frequency F (r) and the operating voltage V (r) and the means 5 for encoding motion 
pictures which effect this encoding by using the processor 1 is destined to encode the 
present frame in Step 5. The target decrease of the electric power consumption can be 

20 attained by causing the peripheral unit including the processor 1 and (or) the local 
decoding memory 6 to be operated constantly with a high frequency when the images are 
in a violent motion (the input image data 101 of the present frame) or to operated 
constantly with a low frequency when the images are in a moderate motion. The means 5 
for encoding motion pictures is fiirther provided with a local decoder possessing the 

25 ftinction of decoding the encoded data 106. Thus, the encoded data 106 of the present 
frame are decoded by the local decoder and stored as the local decoded data 103 in the 
local decoded frame memory 6. The local decoded data 103 of the present frame is used 
when the necessary volume Kp of operation is calculated with respect to the frame which 
will be encoded subsequent to the present frame. The encoded data 106 of the present 

30 fixime is transmitted and accxmiulated in the storage medium via the transmission line. 

(Second failure avoiding step) Fiulher, when the necessary volume Kp of operation 
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calculated by the aforementioned means 2 for calculating the necessary volume of 
operation is smaller than the real necessary volume of operation of the present frame, the 
failure situation arises because the process of the present frame is not completed within the 
duration Te allocated to the operation. Thus, the present system SI is fiimished with a 
5 second failure avoiding means which performs the operation of avoiding the failure 
situation when the operation of encoding the aforementioned one frame is not completed 
within the diuration Te allocated in advance to the operation of encoding or decoding the 
aforementioned one frame. The present mode of embodiment is endowed with the means 
9 for judging completion of the operation as the second failure avoiding means. The 

10 means 9 forjudging completion of the operation, while the means 5 for encoding motion 
pictures is executing the routine for encoding the input image data 101 of the present 
frame at Step 5, interrupts the encoding routine at a prescribed timing, effects a temporary 
suspension during the course of the process, and judges whether or not the operation of 
encoding the present frame is completed. When macro blocks which have not been 

15 encoded exist, the means 9 forjudging completion of the operation judges that 

the necessary volume of operation calculated by the aforementioned means for calculating 
the necessary volume of operation will be smaller than the actual necessary volume of 
operation and that the process will not be completed within the duration T e. Under this 
judgement, the means 5 for encoding motion pictures forcedly processes the relevant 

20 macro blocks as not-coded blocks. Because encoding the relevant macro blocks as 
forcedly not-coded blocks requires quite smaller volume of operation than that of 
encoding the relevant macro blocks as regular macro blocks, it can be possible for the 
means 5 for encoding motion pictures to avoid failure situation and to complete the 
process within the duration Te. 

25 Now, the means 9 for judging completion of the operation will be specifically 

explained below. Fig. 4 shows the relation between the time for effecting the interruption 
and the residual portion of the operation. It is assumed that T e denotes the duration 
allocated to the operation performed on the present frame with the operating frequency F, 
MB the number of macro blocks in one frame, and Ks the necessary volume of operation 

30 required in processing one macro block as an forcedly not-coded block. In all of the frame 
encoding types, the forcedly not-coded block processes are identical with each other, and 
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the volume of operation required for processing a macro block as a forcedly not-coded 
block is far smaller than the volume of operation required for processing a macro block as 
a regular block. The means 9 forjudging completion of the operation calculates the time T 
i for effecting the interruption in accordance with the formula, T i = T e - Ks * MB/F. 
5 The time for effecting the interruption may be calculated by the aforementioned means 3 
for calculating the operating voltage and operating frequency. Then, the means 9 for 
judging completion of the operation effects an interruption in the encoding routine at the 
timing of the time T i, reads out the number MBi of macro blocks which have undergone 
the encoding operation (reference numeral 117) from the register 10 of the number of 

1 0 processed macro blocks, judges whether MBi = MB is satisfied or MBi < MB is satisfied, 
and decides between presence or absence of the completion of the encoding operation. 
When MBi = MB, a designation of the fact that the operation of encoding the present 
frame has been completed, is satisfied, this means 9 finishes the interruption and returns to 
the encoding routine. When MBi < MB, a designation of the fact that the operation of 

1 5 encoding the present frame has not been completed, is satisfied, the means 9 judges that 
the necessary volume of operation calculated by the means 2 for calculating the necessary 
volume of operation is smaller than the actually necessary volume of operation. Under this 
judgement, the means 5 for encoding motion pictures forcedly processes all the macro 
blocks which have not undergone the encoding operation as not-coded macro blocks, and 

20 returns to the encoding routine. The failure situation can be infallibly avoided because the 
volume of operation for processing at least all the macro blocks in a frame as forcedly not- 
coded blocks is secured at the point at which the interruption is effected with the timing of 
the time Ti. 

Incidentally, the failure situation may be avoided by exalting the operating 
25 frequency and the operating voltage of the processor 1 as will be specifically described 
herein below instead of the operation of forcedly not-coded blocks. In this case, the 
interruption is carried out within the duration allocated in advance to the operation of 
encoding the present frame with a timing capable of allowing a time enough to encode all 
the macro blocks still awaiting the encoding operation. 
30 (Proof) 

Now, the fact that the invention of the present patent application can lower the 
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electric power consumption more than the conventional technique which encodes one 
frame while it changes the operating voltage and the operating frequency of the processor 
a plurality of times will be demonstrated below. When a specific volume Kt of operation 
is carried out within a specific duration T t, for example, the decrease of the electric power 
5 consumption can be realized by keeping the operating frequency Ft during the duration Tt 
and setting this frequency Ft as follows. 

F t = Kt/T t ... (Mathematical 6) 
It is assumed, for example, that the operating voltage and the operating frequency of the 
processor 1 are variable in P stages as shown in Fig. 5, the necessary volume of operation 

10 for an one frame is denoted by Kt, and the duration allocated to encoding this frame is 
denoted by T t. On the assumption that the case of setting the operating frequency at F t 
and the operating voltage used for operating the processor 1 with the operating frequency 
F t at VDD and completing the process of the necessary volume Kt of operation within the 
duration T t as shown in Fig. 6 (a) (namely, the operating frequency is fixed) is designated 

15 as Case 1 and the case of setting the operating frequency of the initial value at h*F t and 
the operating voltage used for operating the processor with the operating frequency h*F t 
at V 1, changing the operating frequency of the processor to h*F t/2 afl:er the elapse of the 
time T 1, setting the operating voltage used for operating the processor 1 with the operating 
frequency of h'^'F t/2 at V2, and completing the process of the necessary volume Kt of 

20 operation within the duration of T 1 + T 2 as shown in Fig. 6 (b) (namely, the operating 
frequency is changed once) is designated as Case 2, let us consider the case of encoding 
the aforementioned arbitrary one frame with respect severally to Case 1 and Case 2. One 
and the same volume of operation, namely Kt (cycle), is required for both the cases. 
Meanwhile, the electric power consumed is represented as follows, 

25 P=a*C*f* VDD^ * t ... (Mathematical 7) 

wherein a denotes a coefficient, C denotes the number of transistors of the processor, f 
denotes the operating frequency, VDD denotes the operating voltage, and t denotes the 
operating time. By the calculation in accordance with this formula, the electric power 
consumption Pa of Case 1 and the electric power consumption Pb of Case 2 are found as 

30 follows. 

Pa = a * C * F t * * T t ... (Mathematical 8) 
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Pb = a * C * (h * F t) * Vl^ * T 1 + a * C * (h * F t /2) * V2^ * T2 ... 
(Mathematical 9) 

and 

Pa : Pb = * T t : (h * Vl^ * T 1 + (h/2) * V2^ * T2) ... (Mathematical 10) 
Assume h = 1.5, T 1 = 1/3 * T t, Tb = 2/3 * T t, V = 1, V 1 = 1.5, and V 2 = 0.75, and the 
ratio will be found as follows. 

Pa : Pb= 1^ (1.5 * 1,5^3 + (1.5/2) * 0.75^ * (2/3) « 1 : 1.41 ... (Mathematical 11) 
Thus, the relation Pa < Pb is ascertained. It is clear, therefore, that when a fixed volume of 
operation is processed within a fixed duration, in spite of one and the same volume K t of 
operation, the electric power consumption is lower when the processor is constantly 
operated throughout the time of process with the smallest operating firequency that enables 
the operation to be completed within the duration as in Case 1 than when the operating 
firequency is changed during the time of the process as in Case 2 using the conventional 
technique. It is fiirther clear that this invention which carries out the operation of encoding 
one firame while it is operating the processor 1 with fixed operating voltage and operating 
firequency can lower the electric power consumption than the conventional technique 
which changes the operating voltage and the operating fi"equency a number of times 
during the process of encoding one frame because the operating voltage and the operating 
frequency are decided for each of the blocks involved. 

(Second mode of embodiment) 

Fig. 7 is a schematic block diagram showing the performance of a motion picture 
encoding system S2 of the second mode of embodiment. The motion picture encoding 
system S2 of the present mode of embodiment is derived from the motion picture 
encoding system S 1 of the first mode of embodiment described above by being provided 
as a second failure avoiding means at least with a means 29 for judging the residual 
portion of the volume of operation in the place of the means 9 for judging completion of 
the operation and the processed macro block number resister 10. Fig. 8 is a diagram 
showing a motion picture encoding program Prg 2 and a schematic flow chart of a motion 
picture encoding method which is materialized by that program. The program Prg 2 is a 
program for causing the computer to fiinction as the motion picture encoding system S2 
fiimished with various means and is fiimished with a second failure avoiding step which is 
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executed with an action of interrupting the encoding operation (Step 5). The motion 
picture encoding system S2, unlike the aforementioned motion picture encoding system SI, 
is directed toward solving the problem mentioned above by executing the dynamic control 
of the operating voltage and working frequency which consists in changing the operating 
5 frequency and the operating voltage at which the peripheral unit including the processor 1 
and (or) the local decoded memory 6 operate. Now, the dynamic control of the operating 
voltage and operating frequency will be described in detail below. 

The operating frequency and the operating voltage used for processing the present 
frame are calculated by the means 3 for calculating the operating voltage and operating 

10 frequency based on the numerical values calculated by the means 2 for calculating the 
necessary volume of operation. When the value of the calculated necessary volume Kp of 
operation is smaller than the necessary volume Km of operation actually required for 
processing the present frame, the operating frequency and the operating voltage calculated 
based on the value of the necessary volume Kp of operation is also smaller than the 

15 operating frequency and the operating voltage befitting the actual process of the present 
frame. 

(Second failure avoiding step) The motion picture encoding system S2, like the 
aforementioned motion picture encoding system SI, provides the means 5 for encoding 
motion pictures with N times of equally spaced interrupting points, effects temporary 

20 interruption of the encoding operation, and at the point of interruption, relies on the means 
29 for judging the residual portion of the volume of operation to compare the residual 
portion Ki of operation which is the residual portion of the necessary volume of operation 
of the present frcime calculated by the means 2 for calculating the necessary volume of 
operation with the residual portion of the actual necessary volume of operation during the 

25 operation of encoding the prescribed frame by the means 5 for encoding motion pictures. 
That is, in the first operation of interruption, the means 29 for judging the residual portion 
of operation determines the residual duration T i allocated to the process of the present 
frame and the operating frequency F of the processor 1 and calculates the remaining 
portion Ki of operation in accordance with the formula Ki = Ti * F. Then, the means 29 

30 for judging the residual portion of operation retains the times, Tl, T2, T (i - 1), at 
which the first through (i - 1 )th interruptions are effected and the operating frequencies, F 1 , 
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F2, F(i-l), of the processor at the individual points of interruption and, based on these 
values, calculates the volume Kpm of operation spent in the operation of processing the 
present frame from the point at which the process of the present frame is started till the 
point at which the i'th interruption occurs in accordance with the formula, Kpm = £ Fj * 
5 (T (j + 1) - Tj). FO denotes the operating frequency which was set when the process of 
the present frame was started, and j equals 0, 1, or (i - 1). Then, the means 29 for 
judging the residual portion of the volume of operation judges which equation is satisfied, 
Ki > Kpm * (MB - MBi)/MBi or Ki < Kpm * (MB - MBi)/MBi. The operation of 
interruption is terminated and the process is returned to the encoding routine when the 

10 calculated residual component Ki of the volume of operation and the volume Kpm of 
operation spent in the process of the present frame satisfy the formula, Ki > Kpm * (MB - 
MBi)/MBi. The means 5 for encoding motion pictures continues the process of the 
present frame till the point at which the (i + l)th interruption occurs. The means 29 for 
judging the residual portion of the volume of operation, when the calculated residual 

15 component Ki of the volume of operation and the volume Kpm of arithmetic operation 
spent in the process of the present frame satisfy the formula, Ki > Kpm * (MB - Mbi)/MBi, 
judges that the necessary volume of operation calculated by the means 2 for calculating 
the necessary volume of operation is smaller than the volume of operation actually 
required and gives the means 4 for controlling the operating voltage and operating 

20 frequency an instruction to increase the operating voltage and the operating frequency by 
one step (reference numeral 104). This instruction, when necessary, may be directed 
toward increasing these factors by two or more steps. Incidentally, MB denotes the total 
number of macro blocks contained in the present frame and MBi denotes the number of 
macro blocks which have undergone the encoding operation of the present frame at the 

25 point at which the i-th interruption occurs. Since the provision of the process described 
above enables the operating frequency of the processor to be increased halfsvay along the 
entire course of the process of the present frame, it ensures completion of the process of 
the present frame without entailing the failure situation even when the operating frequency 
set for the processor at the point at which the process of the present frame is started is 

30 smaller than the operation frequency necessary for realizing the volume of operation 
necessary for the process of the present frame. Incidentally, the N points at which the 



f 

• * 

32 

means 5 for encoding motion pictures is interrupted do not need to be separated by equal 
intervals but may be separated by arbitrary intervals. Optionally, the formula Ki > Kpm * 
(BL - BLi)/BLi and the formula Ki < Kpm * (BL - BLi)/BLi may be used in the place of 
the formula Ki > Kpm * (MB - MBi)/MBi and the formula Ki < Kpm * (MB - MBi)/MBi. 
5 In these formulas, BL denotes the total number of blocks contained in the present frame 
and BLi denotes the number of processed blocks of the present frame at the point at which 
the i-th interruption occurs. 

Incidentally, the present system S2 may be provided with the first failure avoiding 
means 1 1 . 

1 0 (Third mode of embodiment) 

The motion picture decoding system S3 according to the third mode of 
embodiment is a system for decoding the encoded motion pictures. Fig. 9 is a schematic 
block diagram showing the performance of the motion picture decoding system S3. The 
motion picture decoding system S3 of the present mode of embodiment is provided with 

1 5 the processor 1 fumished with operating voltages and operating frequencies prepared in r 
steps (r denotes an integer of not less than 2) and enabled to change these operating 
voltages and operating frequencies by a program, the operating voltage and operating 
frequency controlling means 4 for controlling the operating voltage and the operating 
frequency of the processor 1 mentioned above, a local decoding frame memory 36 for 

20 storing the decoded data of the former frame, and a means 39 for judging the residual 
portion of the volume of operating operation and operating on the processor 1. It is 
provided, however, that the local decoding memory 36 may have the operating voltage 
and operating frequency thereof controlled by the means 4 for controlling the operating 
voltage and operating frequency similarly to the processor 1 . The processor 1 is provided 

25 with a means 32 for calculating the necessary volume of operation and operating on the 
processor 1 , the means 3 for calculating the operating voltage and operating frequency and 
operating on the processor 1, and a motion picture decoding means 35 operating on the 
processor 1. The reference numeral 301 denotes an input encoding data, the reference 
numeral 1 02 denotes the designation of the operating voltage and operating frequency, the 

30 reference numeral 105 denotes the supply of the operating voltage and operating 
frequency, and the reference numeral 306 denotes a decoded data and the same reference 
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numerals in the first mode of embodiment denote the parts fiilfilHng identical or 
equivalent functions. Excepting the fact that the system is about decoding system and the 
facts other than the following facts, third mode of embodiment are the same as those of the 
second mode of embodiment. 
5 The performance of the motion picture decoding system S3 will be explained 

below with reference to Fig. 9. Now, the operation of decoding the present frame will be 
explained below on the assumption that an arbitrary one frame to be decoded in all the 
frames decoded sequentially (namely, the frame to be decoded next based on the point at 
which a certain frame was decoded, i.e. the frame not decoded yet at that point and 

10 scheduled to be decoded in the future) is designated as the present frame and one frame 
decoded before the present frame (the frame decoded in the past) is designated as the 
former frame. One and the same process is performed on any one of the frames. The 
motion picture decoding program Prg3 which causes the computer to fimction as the 
motion picture decoding system S3 is substantially the same as the aforementioned motion 

15 picture encoding program Prgl. At Step 5, it causes the computer (more specifically, the 
processor 1 built in the computer) to function as the motion picture decoding means 35 for 
decoding the encoded data of the present frame. The input encoded data 301 which has 
been fed in the motion picture decoding system S3 is fed in the means 32 for calculating 
the necessary volume of operation. The means 32 for calculating the necessary volume of 

20 operation calculates the amount (number of bits) FB of generated information for one 
frame of the encoded data 301 (namely, the encoded data 301 of the present frame) and 
performs a calculation for forecasting the necessary volume Kp of operation. The 
necessary volume Kp of operation is expressed by the following formula. 

Kp = G (FB, MVa, MVv, B, C, BR, Q, AQ, I, E, P) ... (Mathematical 1 1) 

25 In this formula, FB denotes the number of bits of the encoded data of the present frame or 
the former frame, MVa denotes the average value of the size of the vector of motion of the 
present frame or the former frame, MVv denotes the dispersion of the size of the vector of 
motion of the present frame or the former frame, B denotes the number of valid blocks of 
the present frame or the former frame, C denotes the number of valid coefficients of the 

30 present frame or the former frame, BR denotes the bit rate of the present frame or the 
former frame, Q denotes the average value of the quantized step size of the present frame 
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or the former frame, AQ denotes the difference between the average values of the 
quantized step sizes of the present frame and the former frame or the difference between 
the average values of the quantized step sizes of the former frame and the frame preceding 
the former frame, I denotes the frame encoding mode of the choice among I frame, P 
5 frame, and B frame of the present frame, E denotes the volume of operation actually 
required for decoding the former frame, and P denotes the necessary volume of operation 
of the former frame calculated by the means for calculating the necessary volume of 
operation. 

The volume of operation necessary for decoding the present frame depends on the 

10 number of times of the IDCT process, the IQ process, and the VLD process executed in 
decoding the present frame. Then, the number of times of execution of the IDCT process 
depends on the number of valid blocks contained in the present frame and the numbers of 
times of execution of the IQ process and the VLD process depend on the number of valid 
coefficients contained in the present frame. That is, the volume of operation required for 

15 the decoding operation increases (decreases) when the number of valid blocks and the 
number of valid coefficients contained in the present frame are large (small). The 
aforementioned fiinction G, therefore, is so configured that Kp may be set at a large 
(small) value when B and C are large (small). 

The average value MVa of the size of the vector of motion and the dispersion MVa 

20 of the size of the vector of motion are increased (decreased) when the change of images 
between the former frame and the present frame is large (small). In this case, the number 
of valid blocks and the number of valid coefficient in the present frame are increased 
(decreased) and the volume of operation necessary for the encoding operation is increased 
(decreased). The fijnction G mentioned above, therefore, is so configured that Kp may be 

25 set at a large (small) value when MVa and MVv are large (small). 

When the present frame consists of I frame, the volume of operation required for 
the decoding operation decreases because the addition of the forecast image and the 
differential image does not need to be effected in forming the decoded data. The fiinction 
G mentioned above, therefore, is so configured that Kp may be set at a small value when 

30 the present frame consists of one picture. 

The number of valid blocks and the number of valid coefficients are increased 
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(decreased) when the number FB of bits and the frame rate BR of the encoded data is large 
(small). The function G mentioned above, therefore, is so configures that Kp may be set 
at a large (small) value when FB and BR are large (small). The quantized step size has the 
value thereof varied in controlling the bit rate. When Q and AQ are large (small), for 
5 example, the Kp calculated by the function G mentioned above is enabled to approximate 
closely to the volume of operation actually required for the purpose of decoding the 
present frame by paying due respect to the average value Q of the quantized step size and 
the difference AQ of the average value of the quantized step size as by setting the Kp at a 
small (large) value. 

10 Since the motion pictures have a large correlation between the continued frames, 

MVa, MVv, B, C, BR, FB, and Q assume close values between the present frame and the 
former frame. When these parameters are used in the function G mentioned above, 
therefore, their values in the present frame may be used or those in the former frame may 
be used. When the values in the present frame are used, part of the input encoded data 

1 5 which have been received are decoded and the values consequently derived are put to use. 
In this case, the use of the values in the present frame is at an advantage in enabling the 
forecast volume Kp of operation to approximate closely to the necessary volume of 
operation actually required for the decoding operation. When the value in the former 
frame is used, the foreceist volume Kp of operation can be calculated before the input 

20 encoded data of the present frame are received. Thus, the value so used is at an advantage 
in enabling the decoding operation of the received data to be carried out at the same time 
that the input encoded data are received. 

Further, since the motion pictures have a large correlation between the continuing 
frames, the necessary volume of operation required for the decoding operation of the 

25 present frame approximates closely to the value E of operation actually necessary in the 
decoding operation of the former frame. Further, the approximation, P ~ E, arises when 
the forecast volume of operation calculated by the means for calculating the necessary 
volume of operation approximates closely to the value of operation actually required for 
the decoding operation. By duly considering E and P in order that the necessary volume 

30 of operation of the present frame may enable E and P to assume values increased or 
decreased in conformity with the sizes of such parameters as FB, MVa, MVv, B, C, BR, Q, 
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and AQ, the Kp calculated by the function G mentioned above is enabled to approximate 
closely to the necessary volume of operation actually required for decoding the present 
frame. 

The means 32 for calculating the necessary volume of operation may use only one 
5 of these elements or a plurality of such elements in combination. That is, the plurality of 
these elements affect the necessary volume of operation which is required for the 
operation of decoding the present frame. When the means 32 for calculating the necessary 
volume of operation carries out the calculation so as to increase or decreeise the necessary 
volume Kp (cycle) of operation in conformity with these elements, therefore, the 

10 necessary volume Kp of operation calculated by the means 32 for calculating the 
necessary volume of operation approximates closely to the calculated value obtained when 
the decoding operation is actually carried out. 

The means 3 for calculating the operating voltage and operating frequency and the 
means 4 for controlling the operating voltage and operating frequency are the same as 

15 those of the first mode of embodiment. The means 35 for decoding motion pictures 
decodes the input encoded data 305 of the present frame and gives rise to the decoded data 
306. In the decoding operation by the means 35 for decoding motion pictures, the 
decoding operation is performed while the processor 1 is caused by the means 4 for 
controlling the operating voltage and operating frequency to operate with fixed operating 

20 voltage and operating frequency. Since the necessary volume of operation for each frame 
is calculated before the operation of decoding this frame and the frame is decoded while 
the processor is operated with fixed operating frequency and operating voltage befitting 
the necessary volume of operation, the decrease of the electric power consumption can be 
efficiently attained as compared with the conventional technique which changes the 

25 operating frequency and the operating voltage for each of the stated number of blocks 
obtained by dividing the frame. The decoded data 306 is displayed as motion pictures in a 
portable telephone or the image display part of a personal computer or memorized in a 
memory device such as a hard disc. 

The motion picture decoding system S3 is likewise provided with the means 39 for 

30 judging the residual portion of the volume of operation as a second failure avoiding means. 
The means 39 for judging the residual portion of the volume of operation is substantially 
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the same as in the second mode of embodiment mentioned above but is different in respect 
that the judgment is directed toward the calculated volume of the decoding operation and 
not to the calculated volume of the encoding operation. The failure situation can be 
avoided by the means 39 forjudging the residual portion of the volume of operation. As 
in the first mode of embodiment mentioned above, the present mode of embodiment may 
be provided with a means for judging completion of the operation as a first failure 
avoiding means. Incidentally, the decoding operation is not furnished with a means for 
judging completion of the operation and does not perform the forcedly not-coded block 
operation. 

The motion picture encoding system of this invention may be provided with the 
first failure avoiding means 1 1 , the means 9 for judging completion of the operation as the 
second failure avoiding means, and the means 29 and 39 for judging the residual portion 
of the volume of operation as the second failure avoiding means. These components may 
be used either singly or in the form of a proper combination of two or more members. The 
failure avoiding operation may be implemented, for example, by adopting the first and the 
second failure avoiding means all together, and when the failure cannot be avoided by 
increasing the necessary volume of operation by the first failure avoiding means 11, 
increasing the operating voltage and the operating frequency by the means 29 and 39 for 
judging the residual portion of the volume of operation as the second failure avoiding 
means and, when the failure situation still cannot be avoided, simplifying the encoding 
operation by the means 9 for judging completion of the operation as the second failure 
avoiding means. The motion picture encoding program mentioned above may be realized 
with a hard ware which is furnished with the same function as a program. 

(Example 1) 

Example 1 of the motion picture encoding system SI in the first mode of 
embodiment will be explained. A motion picture data formed of 75 frames was used as 
the object for encoding and a 32 firame was chosen as the frame to be encoded. Each 
frame was formed of picture elements arrayed in 144 lines and 176 columns. For the 
encoding operation, MPEG-4 was used. The processor 1 of the motion picture encoding 
system SI was operated with an operating frequency in the range of 189 MHz - 405 MHz 
and an operating voltage in the range of 1.06 V - 1.80 V and adapted to vary in 9 steps 
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separated by intervals of 27 MHz of operating frequency and 0.0925 V of operating 
voltage. 

First, the motion picture encoding system SI secured access to the input frame 
memory 7, acquired the 32 frame, and calculated the necessary volume Kp of operation 
with the aid of the means 2 for calculating the necessary volume of operation. Specifically 
for the calculation of the necessary volume Kp of operation, the sum Z of the sum of 
absolute difference values was calculated in accordance with the following formula using 
the first frame and the 31^^ frame. 

Z = Z |Xij -YijI = 202752 

Then, the amount W of activity of the 32"^ frame which was the present frame was 
calculated in accordance with the following formula. 

Wh in horizontal direction =i:|X(ij) - X(i - 1 J)| = 76032 
Wv in vertical direction =2:|X(i j) - X(i, j-1) = 126720 

Further, the number of times of macro block matching of the former frame M = 
1580, the average quantized step size of the former frame (average value of quantized step 
sizes) Qprev = 4, the number of valid blocks of the former frame B = 399, the number of 
valid coefficients of the former frame C = 6011 the amount of process actually required 
for the encoding of the former frame S = 15447105, and the encoding bit rate of the 
present frame BR = 65536 were obtained. Then, the difference between the average value 
of the quantized step size of the former frame and the average value of the quantized step 
size of the 30^** frame which preceded the former frame by one frame AQpev = -1 was 
calculated. Further, the number of bits actually generated in the former frame D = 56797 
was obtained. Then, the necessary volume of operation Kp was calculated in accordance 
with the following formula using the component elements enumerated above. 

Kp = j + aM + PB + yC + 8Z + sAQprev 

As a result, the necessary volume of operation in Example 1 Kp = 14481056 was 
obtained. 

Further, a calculation for increasing the necessary volume Kp of operation was 
carried out in accordance with the following formula using the aforementioned necessary 
volume of operation calculated from the component elements enumerated above Kp = 
14481056. Here, the aforementioned mathematical 3 will be used for the explanation. 
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Kpf- 14481056 * 1.1 = 15929162 

Then, the operating frequency was calculated in accordance with the following 
formula. 

Fe = KpfiTe = 15929162/(1/15) =239 MHz 
By calculating F (r) which satisfied F(r) > Fe and F(r-l) < Fe as well, an operating 
frequency 243 MHz was selected as the operating frequency satisfying these relations in 
all the operating frequencies variable in the nine steps of the processor land an operating 
voltage of 1.25 V was selected as well. The means 4 for controlling the operating voltage 
and operating frequency was given an instruction directed toward enabHng at least the 
processor 1 to operate with the operating frequency F (r) = 243 MHz and the operating 
voltage V (r) = 1 .25 V befitting the frequency. The means 4 for controlling the operating 
voltage and operating frequency effected a control aimed at causing at least the processor 
1 to operate constantly with an operating voltage of 243 MHz and an operating frequency 
of 1 .25 V. The means 5 for encoding the motion pictures acquired the fi-ame F fi-om the 
input frame memory 7 and performs an encoding operation and gave rise to encoded data 
with the air of the processor 1 which was in a state of being constantly operated with the 
operating frequency 243 MHz and the operating voltage of 1 .25 V mentioned above. 

Further, while the encoding routine was being executed, the means 9 for judging 
completion of the operation calculated the time for interruption in accordance with the 
following formula and effected necessary interruption. 

Ti = Te - Ks * MB/F 

= 0.06666 - 37 * 99/(243000000) 
~ 0.06665 

Further, the means 9 for judging completion of the operation judged whether or not 
MBi < MB was satisfied at this interrupting timing. In Example 1 , since MBi < MB was 
satisfied at the timing of Ti = 0.06665 £ind the operation of encoding the present frame was 
not completed, all the remaining macro blocks are processed as forcedly not-coded macro 
block and returned to the encoding routine. 

(Example 2) 

Example 2 dealing with the motion picture encoding system S2 of the second 
mode of embodiment will be explained below. In Example 2, this system was configured 
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so as to effect interruption four times in the encoding operation. The means 29 forjudging 
the residual portion of the volume of operation calculated Ki = Ti * F and Kpm = SFj * 
(T(j + 1) - Tj) respectively at the points of the first and the second interruption, calculated 
the residual portion of the actually necessary volume of operation Kpm * (MB - MBi), and 
5 judged the choice between Ki >Kpm * (MB - MBi)/MBi and Ki < Kpm * (MB - 
MBi)/MBi. In Example 2, since Ki >Kpm * (MB - MBi)/MBi was satisfied the process 
completed the interrupting operation and caused the means 5 for encoding the series 
images to continue the encoding operation till the third interruption. The same calculation 
and judgment were carried out during the third interruption which was the next 
10 interruption. Since Ki < Kpm * (MB - MBi)/MBi was satisfied in Example 2, the means 4 
for controlling the operating voltage and operating fi'equency was given an instruction 
directed toward increasing the operating fi^equency and the operating voltage by one step 
respectively to a fi-equency Fp + 1 = 270 MHz and a voltage Vp +1 = 1 .34 and using them 
as the operating fi-equency and the operating voltage. 

15 

Industrial Applicability 

The motion picture encoding or decoding system and the motion picture encoding 
or decoding method according to this invention perform a calculation for forecasting a 
necessary volume of operation required for encoding or decoding a present fi*ame yet to 

20 be encoded or decoded (a fi'ame to be encoded or decoded in the fizture), controls the 
present fi*ame substantially constantly with the smallest possible operating fi*equency 
within the duration allocated to the process of the present fi-ame, consequently controls 
dynamically the operating voltage and operating frequency in the unit of frames, and 
realizes the decrease of the electric power consumption as described above. 

25 Since they are furnished with a failure avoiding means, the failure situation which 

occurs when the calculated necessary volume of operation is smaller than the volume of 
operation actually required can be avoided the motion pictures which have undergone the 
encoding or decoding operation can be prevented from being deteriorated. 
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